Wearable article supporting performance capture equipment

ABSTRACT

Implementations provide a wearable article for a performance capture system. In some implementations, a wearable article includes one or more regions, where the one or more regions are configured to be worn on at least a portion of a body of a user, and where at least one of the one or more regions are configured to hold performance capture equipment in predetermined positions. In some implementations, the wearable article also includes a plurality of mounting mechanisms coupled to the one or more regions for mounting reference markers to be used for position determination. In some implementations, the wearable article also includes a plurality of fastening mechanisms coupled to the one or more regions for fastening devices and accessories for controlling the reference markers.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application a continuation-in-part of U.S. patent application Ser. No. 17/246,501 (WD0063US1), entitled “WEARABLE ARTICLE FOR A PERFORMANCE CAPTURE SYSTEM,” filed Apr. 30, 2021, which claims priority from U.S. Provisional Patent Application No. 63/066,038 (WD0063PP1), entitled “WEARABLE ARTICLE FOR A PERFORMANCE CAPTURE SYSTEM,” filed Aug. 14, 2020, and is related to U.S. patent application Ser. No. 17/246,508 (WD0063US2), entitled “WEARABLE ARTICLE WITH CONDUITS FOR A PERFORMANCE CAPTURE SYSTEM,” filed Apr. 30, 2021, which claims priority from U.S. Provisional Patent Application No. 63/066,046, entitled “WEARABLE ARTICLE WITH CHANNELS FOR A PERFORMANCE CAPTURE SYSTEM,” filed Aug. 14, 2020, which are hereby incorporated by reference as if set forth in full in this application for all purposes.

BACKGROUND

The present disclosure generally relates to a performance capture system, and more specifically to a wearable article for a performance capture system. Many visual productions (e.g., movies, videos, clips, and recorded visual media) include combinations of real and digital images to create animation and special effects that form an illusion of being integrated with live action. For example, a visual production may include a live actor in a location shoot appearing in a scene with a computer-generated (“CG,” “virtual,” or “digital”) character. It is desirable to produce seemingly realistic visual productions by compositing CG items with the live action items. Cameras typically track reference markers on a suit as an actor moves, and the cameras relay the movements of the actor to a system. Artists or animators may then create a digital character from such movements, and the digital character is then incorporated into a film.

SUMMARY

Implementations generally relate to a wearable article for a performance capture system. In some implementations, a wearable article includes one or more regions, where the one or more regions are configured to be worn on at least a portion of a body of a user, and where at least one of the one or more regions are configured to hold performance capture equipment in predetermined positions. In some implementations, the wearable article also includes a plurality of mounting mechanisms coupled to the one or more regions for mounting reference markers to be used for position determination. In some implementations, the wearable article also includes a plurality of fastening mechanisms coupled to the one or more regions for fastening devices and accessories for controlling the reference markers.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a front-view diagram of an example wearable article for a performance capture system, which may be used for implementations described herein.

FIG. 2 is an image of an example fastening mechanism on a wearable article, according to some implementations.

FIG. 3 is a front-view image of example fastening mechanisms on a wearable article, according to some implementations.

FIG. 4A is a diagram of an example adjustment mechanism for a wearable article, where the adjustment mechanism is in a partially tightened position, according to some implementations.

FIG. 4B is a diagram of an example adjustment mechanism, where the adjustment mechanism is in a fully tightened position, according to some implementations.

FIG. 5 is a front perspective-view diagram of an example wearable article for a performance capture system, which may be used for implementations described herein.

FIG. 6 is a diagram of an example reference marker assembly, according to some implementations.

FIG. 7A is an exploded diagram of an example mounting mechanism assembly 500, according to some implementations.

FIG. 7B is a diagram of a mounting mechanism assembly, where a fastener is coupled to a reference marker housing, according to some implementations.

FIG. 8 is a rear perspective-view diagram of an example wearable article for a performance capture system, which may be used for implementations described herein.

FIG. 9 is a block diagram of an exemplary computer system, which may be used for implementations described herein.

FIG. 10 is a block diagram of an example visual content generation system, which may be used to generate imagery in the form of still images and/or video sequences of images, according to some implementations.

FIG. 11 is a block diagram of an example computer system, which may be used for implementations described herein.

DETAILED DESCRIPTION

Implementations generally relate to a wearable article for a performance capture system. The wearable article may be referred as a motion capture (mocap) suit. As described in more detail herein, the mocap suit is designed to be setup quickly and to allow an actor to put the suit on without much preparation by the actor. The suit was also designed to help the actor get into character by minimizing distractions from reference markers worn on the body of the actor.

In some implementations, a wearable article includes one or more regions, where the one or more regions are configured to be worn on at least a portion of a body of a user, and where at least one of the one or more regions are configured to hold performance capture equipment in predetermined positions. In some implementations, the wearable article also includes a plurality of mounting mechanisms coupled to the one or more regions for mounting reference markers to be used for position determination. In some implementations, the wearable article also includes a plurality of fastening mechanisms coupled to the one or more regions for fastening devices and accessories for controlling the reference markers.

FIG. 1 is a front-view diagram of an example wearable article 100 for a performance capture system, which may be used for implementations described herein. In this example, wearable article 100 or mocap suit may also be referred to as a hero suit, which is worn by an actor on a movie set. Other implementations of the wearable article are described in more detail herein.

In various implementations, wearable article 100 includes one or more regions. In various implementations, the one or more regions are configured to be worn on at least a portion of a body of a user. In various implementations, the one or more regions have a first pliability and a second pliability, where the first pliability and the second pliability are different pliabilities. In various implementations, at least one of the one or more regions are configured to hold devices in predetermined positions while maintaining shape and respective pliability. Various implementations directed these features are described in more detail herein.

In various implementations, wearable article 100 may include multiple regions. In some implementations, the regions may be integrated into one single piece. In some implementations, some regions may be separate pieces of the wearable article. The separate pieces may be referred to as regions of the wearable article or portions of the wearable article. In some implementations, the wearable article may include multiple mounting mechanisms coupled to the one or more regions for mounting one or more reference markers to be used for position determination. In various implementations, the one or more regions include at least one first region having the first pliability, and at least one second region having the second pliability, where the second pliability is greater than the first pliability, and where the at least one second region is coupled to the at least one first region and configured to allow movement of the user. In some implementations, the one or more regions include at least one adjustment mechanism coupled to at least one first region of the one or more regions, where the at least one adjustment mechanism enables the at least one first region to expand or contract. In some implementations, the one or more regions include one or more torso portions, one or more arm portions, and one or more leg portions. In some implementations, the one or more regions include one or more underarm portions and one or more crotch portions. In some implementations, the one or more reference markers include an active reference marker. In some implementations, the one or more reference markers include a passive reference marker. In some implementations, each mounting mechanism of the plurality of mounting mechanisms includes a through hole for passing through at least a portion of a reference marker. In some implementations, each mounting mechanism of the plurality of mounting mechanisms includes a fastener for securing a reference marker. In some implementations, the fastener includes a securing ring attached to a portion of a passed-through reference marker. In some implementations, the securing ring includes a predetermined color selected for imaging identification. Various implementations directed these features are described in more detail herein.

In some implementations, wearable article 100 includes one or more regions of the wearable article configured to be worn on at least a portion of a body of a user. In some implementations, at least one of the one or more regions including at least one base layer and at least one secondary layer configured to form at least one connection passage between the at least one base layer and the at least one secondary layer. The terms connection passage, connection conduits, and conduits may be used interchangeably. In some implementations, the at least one connection passage is configured to provide access for flexible cable connections between at least one reference marker and one or more other reference markers or a control unit. In some implementations, the at least one connection passage is configured to allow movement of a flexible cable within the connection passage in response to movement of the user. Various implementations directed these features are described in more detail herein.

In some implementations, the one or more regions include at least one first region having a first pliability. In some implementations, at least one second region having a second pliability, where the second pliability is greater than the first pliability, where the at least one second region is coupled to the at least one first region and configured to allow movement of the user. In some implementations, the at least one connection passage is flexible. In some implementations, the at least one connection passages is curved. In some implementations, the at least one connection passage enable movement of communication connections that connect a reference marker to at least one other reference marker or control box. In some implementations, each connection passage of the at least one connection passage includes one or more mounting mechanisms for mounting the at least one reference marker to be used for position determination. In some implementations, each mounting mechanism includes a through hole for passing through at least a portion of a reference marker. Implementations described herein solve various issues. For example, the connection passages allow connection between devices and allows movement of the user that could be impinged by the cable connections. Also, the cable connections may provide for more than electrical connection. Various implementations directed these features are described in more detail herein.

In various implementations, wearable article 100 includes two or more support portions having a predetermined pliability. As shown, the support portions are configured to be worn on at least a portion of a body of a user. For example, as shown, the support portions may include a torso portion 102, arm portions 104 and 106, leg portions 108 and 110, etc. For example, in some implementations, there may be two arm portions and two leg portions.

In various implementations, the support portions have some pliability, yet are sufficiently firm to maintain shape and to support devices such as reference markers and electronics associated with a performance capture system. Having some pliability enables the support portions to provide fit and comfort for the user. Also, having some stiffness or firmness enables the support portions to hold devices such as reference markers and electronics. The pliability and other features of the support portions are described in more detail below.

In some implementations, wearable article 100 may be divided into separate upper and lower sections. For example, the upper section may include torso portion 102 and arm portions 104 and 106, thereby resembling a long sleeved top. Also, lower section may include leg portions 108 and 110, thereby resembling pants. The number of portions for each of the upper and lower sections may vary, and will depend on the particular implementation. In some implementations, the upper and lower sections may be removably attachable by any suitable attaching mechanisms such as hook and loop material (e.g., VELCRO™), one or more zippers, buttons, snaps, etc. In some implementations, the upper and lower sections may be integrated into a single piece, in which case wearable article 100 may include an opening and closing mechanism such as a zipper (e.g., see FIG. 2A) in order to enable a user to put wearable article 100 on or take wearable article 100 off.

As described in more detail herein, each of the support portions includes one or more mounting mechanisms for mounting one or more reference markers to be used for position determination. In various implementations, a reference marker may be any device or physical object that a camera can capture in an image or video, where a system such as a live action capture system can compute a position of the reference marker relative to the camera and/or relative to one or more other physical objects in the live action scene.

Wearable article 100 also includes one or more pliable portions having a predetermined pliability. In various implementations, the one or more pliable portions may include underarm portions 112 and 114, and a crotch portion 116. For example, in some implementations, there may be two underarm portions and one crotch portion. In some implementations, the pliable portions may also include elbow and knee portions (not shown), etc. The particular number and the particular locations of the pliable portions may vary and will depend on the implementations.

In some implementations, the pliable portions meet a predetermined pliability, and the support portions meet a second predetermined pliability. In various implementations, the predetermined pliability of the pliable portions is different from the predetermined pliability of the support portions. In various implementations, the pliability of the pliable portions is greater than the pliability of the support portions (e.g., the support portions are stiffer than the pliable portions). In various implementations, each pliable portion is coupled between two or more support portions and is configured to allow movement of the user.

In various implementations, the pliability of a portion of a wearable article is a general measure of how easy or difficult it is for a user to move that portion of the wearable article. The greater the pliability, the easier it is for a user to move that portion. Movement of a portion may include movements such as stretching, bending, compressing, or any other change to the position, configuration, or state of the portion. Such movements also enable connection conduits and wires within the connection conduits to move similarly. The terms connection passage, connection conduits, and conduits may be used interchangeably. Example implementations directed to the movement of connection conduits and/or wires are described in more detail herein. In some implementations, pliability is the inverse of stiffness and either may be referred to herein to describe a property or behavior of the material of a portion of the wearable article.

In various implementations, two different types of materials may be used for the support portions. The number of different material types may vary, depending on the particular implementation. For example, in other implementations, more than two types of materials may be used. Where two different types of materials are used in a wearable article, they are referred to as support portions and pliable portions with each portion type having the same, or approximately the same, pliability.

Any suitable material may be used for the various portions. For example, portions may be made of neoprene, nylon, Lycra™, unbroken loop (UBL) Lycra™, etc. The particular material may vary, and will depend on the particular implementation. In some implementations, layers of the same material type or a combination of different material types may be used to achieve a desired pliability.

As shown, in various implementations, each of the one or more support portions may include one or more adjustment mechanisms 118 and 120 (demarcated with dashed lines). In various implementations, the adjustment mechanisms are used for mounting one or more reference markers to be used for position determination. The adjustment mechanisms enable one or more support portions to expand or contract. In this particular example scenario, adjustment mechanisms 118 and 120 are unengaged or fully untightened. Example implementations directed to adjustment mechanisms are described in more detail below in connection with FIGS. 4A and 4B.

Also shown are various reference markers 130, 132, 134, 136, 138, 140, 142, 144, 146, and 148. Reference markers 130 - 148 are distributed around different portions of wearable article 100. In various implementations, each reference marker 130 - 148 protrudes from the outer surface of wearable article 100. While a particular number of example reference markers are shown, the number of reference markers may vary, and the actual number depends on the particular implementation.

In various implementations, each of reference markers 130 - 148 is connected to one or more other reference markers and/or connected to a control box or control unit via communication links such as wires 150, 152, 154, 160, 162, and 164. The phrases control box or control unit may be used interchangeably. Wires 150 - 164 provide power and signals to and from reference markers 130-148. While a particular number of example wires are shown, the number of wires may vary, and the actual number depends on the particular implementation.

In various implementations, wearable article 100 includes connection channels or connection conduits 170, 172, 174, 180, 182, and 184 through which respective wires 150, 152, 154, 160, 162, and 164 pass. The phrases connection channels or connection conduits may be used interchangeably. Wires 150-164 are shown as dashed lines indicating that they are hidden inside respective connection conduits 170-184. Also, reference markers 130-148 fit inside a given connection conduit, and each reference marker 130-148 partially protrudes through the outer surface of the given connection conduit of wearable article 100. In various implementations, each of the connection conduits couples at least one reference marker to one or more other reference markers. In various implementations, the connection conduits may connect the one or more reference markers to a control unit such as control unit 612 of FIG. 6 below.

In various implementations, each connection conduit has one or more mounting mechanisms, where each reference marker protrudes through a mounting mechanism. In some implementations, a mounting mechanism may be a hole in the outer surface of a given connection conduit. Example implementations directed to reference markers are described in more detail herein, in connection with FIGS. 6 and 7, for example.

In various implementations, one or more connection conduits may include at least one inner layer and at least one outer layer. In various implementations, one or more connection conduits may include at least one inner layer, at least one outer layer, and a space between the at least one inner layer and at least one outer layer. In other words, the space is sandwiched between the inner layer and the outer layer. The space enables movement of a wire in the space, where the wire connects a reference marker to at least one other reference marker. For example, the movement of the wire may be lateral movement or side-to-side lateral movement. In some implementations, the width of at least some of the connection passages are wider than the height of those connection passages in order to allow for lateral movement (e.g., side-to-side movement, etc.). For example, in various implementations, the movement of the wire may also follow the contour of the body of the user, where the body contour of the user shapes the connection conduit, which in turn may influence the shape the wire. The space enables the wire to move independently from the connection conduit.

In various implementations, the outer surface of the given connection conduit is the outer surface of an outer layer through which a reference maker protrudes. In various implementations, a given connection conduit has an inner layer that rests against the body and/or skin of the user. In various implementations, the outer layer of a given connection conduit couples to the inner layer such that the outer layer is positioned on top of the inner layer, where the borders of the outer layer and the inner layer are joined, thereby forming a conduit. As such, reference markers may be positioned the connection conduit, and wires may pass through the connection conduit.

In various implementations, the size of connection conduits 170-184 may vary depending on the particular implementation. For example, in various implementations, the width of each connection conduit 170-184 is at least as wide as the physical diameter of reference markers 130-148 (e.g., 0.25 inches, 0.5 inches, etc.). In various implementations, some connection conduits such as connection conduits 172 and 182 may be narrower than other connection conduits such as connection conduits 170 and 180. The narrower width keeps such connection conduits at a lower profile and less bulky. This is particularly useful at portions of wearable article 100 such as at the wrists where the surface area is smaller and where the wearable article 100 may fit more tightly.

In various implementations, some connection conduits such as connection conduits 174 and 184 may be wider than other connection conduits such as connection conduits 170 and 180. This allows for more wires to pass through and more reference points to be placed in relation to a single conduit. This is particularly useful at portions of wearable article 100 such as at the torso or legs where the surface area is larger.

In various implementations, the length of each connection conduit 170-184 may vary, and may extend as appropriate to enable placement of reference markers. While a particular number of example connection conduits are shown, the number of connection conduits may vary, and the actual number depends on the particular implementation. In various implementations, connection conduits 170-184 may be distributed over the front, back, and sides of wearable article 100. The particular placement of each connection conduit 170-184 may vary and depends on the particular implementation.

In various implementations, wearable article 100 has one or more fastening mechanisms such as fastening mechanism 190 attached thereto. In this example implementation, fastening mechanism 190 is a flap that is positioned on the torso portion 102 of wearable article 100. As shown, fastening mechanism 190 is configured to be partially and detachably placed over the front zipper 192 of wearable article 100. In various implementations, fastening mechanism 190 may be partially removed in order to allow the user full access to zip up zipper 192 in order to put wearable article 100 on. Conversely, fastening mechanism 190 may be partially removed in order to allow the user to fully unzip zipper 192 in order to take torso portion 102 off. Further implementations directed to fastening mechanism are described in more detail herein such as in connection with FIG. 2, for example.

In some implementations, fastening mechanism 190 includes a strap 194 for fastening a device such as a controller device 196 to wearable article 100. As shown, controller device 196 is partially visible and partially exposed thereby facilitating the user in inserting controller device 196 underneath strap 194. As such, strap 194 fastens controller device 196 to wearable article 100. In various implementations, controller device 196 is configured for controlling the reference markers such as reference markers 130-148.

While one fastening mechanism 190 is shown, there may be multiple fastening mechanisms attached to wearable article 100. Such fastening mechanisms may be positioned at different portions of wearable article 100. For example, fastening mechanism 190 is positioned in the front of wearable article 100. There may be other fastening mechanisms located at other regions of wearable article 100 such as in the back region of wearable article 100. Other example implementations of fastening mechanism 190 are described in more detail herein, in connection with FIG. 3, for example.

In some implementations, the outer layer may have a different thickness from the inner layer. For example, in some implementations, the outer layer may be thicker than the inner layer. This enables a thinner inner layer against the body and/or skin of the user to vary for comfort to the user (e.g., fit, temperature control, etc.). This also enables the thicker outer layer to be firmer in order to facilitate in supporting the contents of the connection conduit (e.g., wires, reference markers, etc.). This also enables the thickness of the outer layer to support the mounting mechanism to be an appropriate thickness to support reference markers. In some implementations, the thicknesses of the outer and inner layers may be the same.

In some implementations, the inner layer of a connection conduit may be the same material as the support layer, where a portion of the support layer functions as the inner layer. Alternatively, the inner layer may be coupled to (e.g., sown to) the support layer. For example, in some implementations, the inner layer may be coupled to the support layer contiguously to replace a portion of the support layer. In some implementations, the inner layer may be coupled to the support layer such that the inner layer is positioned on top of a portion of the support layer.

In some implementations, one or more of the connection conduits may have an outer layer and an inner layer having different thicknesses. For example, the inner layer may have a thickness that is appropriate for comfort and/or support. The outer layer may have a thickness that is appropriate for the mounting mechanisms. Also, the thickness of the outer layer a given connection conduit and that of the inner layer of the given connection conduit may be different from those of other connection conduits, depending on their particular locations in wearable article 100. For example, the thickness of the outer and inner layers of a connection conduit at the forearms of wearable article 100 may be thinner than the thickness of the outer and inner layers of a connection conduit at the torso of wearable article 100. Also, the thickness of the outer and inner layers of any one or more of the connection conduits may be different from or may match the thickness of other layers of a support portion or pliable portion of wearable article 100, depending on the particular implementation.

FIG. 2 is an image 200 of an example fastening mechanism 202 on a wearable article 204, according to some implementations. In various implementations, fastening mechanism 202 is a flap that is positioned on the torso portion of wearable article 204, similar to fastening mechanism 190 of FIG. 1 that is positioned on the torso portion 102 of wearable article 100. As shown, fastening mechanism 202 is configured to be detachably placed over a front zipper 206 of wearable article 204.

As shown, fastening mechanism 202 has an edge 210 that is attached or fixed to wearable article 204. In this example, edge 210 is sown or stitched to wearable article 204. Edge 210 may be coupled to wearable article 204 by other means such as by a zipper, VELCRO^(TM), snaps, buttons, etc. An advantage of at least edge 210 being sown or stitched to wearable device 204 in a substantially permanent manner is that it ensures that the fastening mechanism 202 and any equipment fastened by fastening mechanism 202 remain secured to wearable article 204.

Fastening mechanism 202 also has an edge 212 that is detachably coupled to wearable article 204. In this example, edge 212 is detachably coupled to wearable article 204 with a zipper 214. Edge 212 may be detachably coupled to wearable article 204 by other detachable means such as by zipper, VELCRO™, snaps, buttons, etc. An advantage of edge 212 being coupled to wearable article 204 by zipper 214 is that zipper 214 can be conveniently and quickly zipped and unzipped. Also, when zipped, zipper 214 keeps fastening mechanism 202 securely attached at edge 212. Other detachable means such as VELCRO™, snaps, buttons, etc. would take longer to operate than zipper 214.

In various implementations, fastening mechanism 202 is partially detachable (e.g., by zipper 214 being unzipped) in order to enable zipper 206 to be fully exposed. For example, when zipper 214 is unzipped, fastening mechanism 202 can be peeled or moved away from the torso of the user thereby fully exposing zipper 206. Exposing zipper 206 enables the user to fully zip up zipper 206 when putting wearable article 100 on. Conversely, when zipper 214 is unzipped after use of wearable article 204 and peeled away, zipper 206 becomes fully accessible to the user. This enables the user to fully unzip zipper 206 in order to take the torso portion of wearable article 204 off.

Fastening mechanism 202 is shown to have zipper 214 on the left side (facing the torso), as shown. Alternatively, zipper 214 may instead be on the right side (facing the torso). Also, in some implementations, zipper 214 may be replaced with another type of fastener such as VECRO™, buttons, etc. An advantage of a zipper is that a zipper is quick to zip and unzip and secure.

As shown, in various implementations, fastening mechanism 202 includes a main body of a fastening strap 220, or strap 220. In some implementations, fastening mechanism 202 includes one or more strap extension portions having one or more quick-release mechanisms coupled thereto such as a snap 222 and a snap 224. Snaps 222 and 224 enable strap 220 to fasten a device such as device 230. As shown, when snaps 222 and 224 are snapped in place, the strap extension portions are sufficiently long in order to enable strap 220 to form a pocket into which device 230 is inserted. Strap 220 allows a sufficient space for a device such as device 230 to be securely seated behind strap 220.

In this example implementation, device 230 is a controller device that controls signals to reference markers such as reference markers 240 and 242. While fastening mechanism 202 is shown in this example as fastening a controller device, fastening mechanism 202 may also fasten other types of devices such as battery pack, etc.).

In various implementations, strap 220 has a substantially triangular shape. As shown, strap 220 has longer sides at the bottom and at least one size vertical side to provide support and to secure device 230 at those supporting sides. As shown, the other vertical side and the top portions of strap 220 are open to enable device 230 to be conveniently and quickly inserted or extracted from strap 220. As such, the triangular shape of strap 220 facilitates the user in inserting controller device 230 behind strap 220 and extracting controller device 230 from behind strap 220.

In some implementations, multiple strap extensions such as the two shown are used in order to accommodate portions of a device and/or accessories that do not fit in strap 220 and that protrude out of strap 220. This allows for more openings and for some flexibility of the structure of strap 220 to accommodate parts of device 230 or other parts connected to device 230 to protrude away from strap 220. For example, device 230 may have an antenna, connectors, and cables that extend out from behind strap 220, as shown.

In some implementations, strap 220 includes a portion that is a mesh material. The mesh material enables a user to conveniently and quickly see some of device 230. This also enables the user to see any identifying information (e.g., serial code, etc.) on the surface of the device that is visible through the mesh. In various implementations, where strap 220 includes a mesh portion, as shown, the mesh portion enables devices that may need air intake for cooling (e.g., using airflow, internal fans, etc.) to minimize any overheating.

While one fastening mechanism 202 is shown, there may be multiple fastening mechanisms attached to wearable article 204. Such fastening mechanisms may be positioned at different portions of wearable article 204. For example, fastening mechanism 202 is positioned in the front of wearable article 204. There may be other fastening mechanisms located at other regions of wearable article 204 such as in the back region of wearable article 204. Other example implementations of fastening mechanisms are described in more detail below, in connection with FIG. 3, for example.

FIG. 3 is a front-view image 300 of example fastening mechanisms 302 and 304 on a wearable article 306, according to some implementations. In various implementations, fastening mechanisms 302 and 304 are flaps that are positioned on the torso portion of wearable article 306, similar to fastening mechanism 202 of FIG. 2 being positioned on the torso portion of wearable article 204.

In various implementations, fastening mechanisms 302 and 304 have respective edges that are attached or fixed (e.g., sown, stitched, etc.) to wearable article 306 and have respective edges that are detachably coupled to wearable article 306 by respective zippers 308 and 310. In other implementations, one or more of zippers 308 and 310 may be substituted with other coupling means such as by VELCRO™, snaps, buttons, etc. Fastening mechanisms 302 and 304 are shown to have zippers 308 and 310 at the top, as shown. Alternatively, zippers 308 and 310 may instead be on other sides of fastening mechanisms 302 and 304.

Fastening mechanism 302 includes a pocket 312 and a strap 314 that holds or fastens a device 316 to fastening mechanism 302. Pocket 312 may have an opening for nesting device 316 and may have other smaller openings to enable portions of device 316 and/or accessories attached to device 316 to protrude out of pocket 312.

Fastening mechanism 304 also includes a pocket and a strap extension (not labeled for ease of illustration) that hold or fasten a device (not labeled) to fastening mechanism 304. As shown, the pocket may have an opening for nesting the device and may have other smaller openings to enable portions of the device and/or accessories attached to the device to protrude out of the pocket. For example, sets of cables 318 and 320 are allowed to protrude from the pocket in order for the cables to travel to various portions of wearable article 306 to feed power and/or signals to various reference markers.

In various implementations, wearable article 306 also includes fastening loops such as fastening loop 322 that secures cables such as cable 324 and fastening loop 326 that secures cables such as cable 328. As shown, fastening loops 322 and 326 have snaps that enable them to open to conveniently receive one or more cables and snap into place to secure such cables. The number and locations of such fastening loops positioned around wearable article 306 may vary, depending on the particular implementation.

FIG. 4A is a diagram of an example adjustment mechanism 400 for a wearable article, where adjustment mechanism 400 is in a partially tightened position, according to some implementations. Adjustment mechanism 400 may be used to implement adjustment mechanisms 118 and 120 and/or other adjustment mechanisms of wearable article 100 of FIG. 1 and/or other wearable article implementations described herein.

In this particular example implementation, adjustment mechanism 400 is a zipper having teeth 402, a slider 404, and a pull-tab 406. As shown, teeth 402 are partially engaged or zipped up. As such, adjustment mechanism 400 is partially tightened, as shown.

In various implementations, the two rows of teeth are attached to the material of a support portion, such as adjustment mechanisms 118 and 120 shown in FIG. 1. The two rows of teeth may be positioned in parallel to each other. This enables slider 404 of FIG. 4A to be pulled by a user grasping pull-tab 406 and pulling (here upward) in order to engage and pull the two rows of teeth together as shown. Adjustment mechanism 400 enables a user to tighten it as much as desired, as well as to minimize bagginess in the mocap suit. Adjustment mechanism 400 may be designed to operate nearly silently to avoid generating noise. For example, a benefit of adjustment mechanisms 118 and 120 being zippers is that they are relatively quiet when adjusted, which is much quieter than hook and loop material, such as Velcro™.

FIG. 4B is a diagram of adjustment mechanism 400, where adjustment mechanism 400 is in a fully tightened position, according to some implementations. Shown are teeth 402, slider 404, and pull-tab 406. As shown, teeth 402 are fully engaged or zipped up. As such, adjustment mechanism 400 is fully tightened.

The particular material used for adjustment mechanism 400 may vary, depending on the particular implementation. For example, adjustment mechanism 400 may be plastic, metal, or another material.

While adjustment mechanism 400 is a zipper mechanism, the adjustment mechanism for any given support portion of wearable article 100 of FIG. 1 and/or other wearable article implementations described herein may vary depending on the particular implementation. For example, a given adjustment mechanism may involve Velcro, snaps, buttons, etc.

FIG. 5 is a front perspective-view diagram of an example wearable article 500 for a performance capture system, which may be used for implementations described herein. As shown, wearable article 500 includes support portions, including a torso portion 502, arm portions 504 and 506, leg portions 508 and 510, etc. Wearable article 500 also includes pliable portions, including underarm portions such as underarm portion 514, and including a crotch portion 516.

Also shown are various reference markers that are distributed around different portions of wearable article 500, such as reference markers 520, 522, 524, etc. As shown, in some implementations, each reference marker protrudes from the outer surface of the wearable article. In various implementations, some reference markers may be covered or hidden. Being covered makes the reference markers less distracting to an actor, as well as more aesthetically pleasing. As such, being covered may help the actor to be more in character when acting. Furthermore, being covered helps reference markers to be more stable on the mocap suit and less likely to be pulled or pushed out of place.

In various implementations, wearable article 500 main include auxiliary or supplemental support portions to provide support for additional reference markers. For example, shown is a head portion 530, hand portions 532 and 534, and feet portions 536 and 538. In some implementations, various reference markers such as reference markers 540, 542, and 544 may be distributed around these portions.

In various implementations, head portion 530 may be any type of article worn on the head of a user. For example, head portion 530 may be a headband, a hat, a hair clip, etc. Hand portions 532 and 534 may be any type of articles worn on the hands. For example, hand portions 532 and 534 may be hand gloves, finger gloves, bracelets, rings, etc. Feet portions 536 and 538 may be any type of articles worn on the feet. For example, feet portions 536 and 538 may be shoes, anklets, rings, etc. The particular type of articles worn on the head, hands, or feet, or any other part of the body may vary and will depend on the particular implementation.

Also shown are communication links such as wires 550 and 552 that connect to the various reference markers. Wires 550 and 552 provide power and signals to and from the reference markers.

FIG. 6 is a diagram of an example reference marker assembly 600, according to some implementations. As shown, reference marker assembly 600 includes a reference marker housing 602 and a power and or communication connection 604. In various implementations, reference marker housing 602 contains a reference marker 606 (location signal source), which may include a light emitting diode (LED), radio frequency identification (RFID) tag or other transponder, etc. In various implementations, reference marker assembly 600 may represent any reference marker attached to any of the wearable articles shown in FIGS. 1, 5, and 8.

In this example, the reference marker in reference marker housing 602 is an active reference marker. Reference markers may be active or passive, depending on the particular implementation. In various implementations, an active reference markers have electronics that recognize coded signals and activate LEDs. Passive reference markers are retroreflective in that they reflect incoming infrared (IR) radiation into the direction of an incoming light, thereby directing a light beam back to its source.

In some implementations, reference marker housing 602 is translucent or semi-translucent in order to improve transmission of signals to and from reference marker 606. In addition, reference marker housing 602 may be designed to focus and/or direct the signal to and from reference marker 606.

In some implementations, reference marker housing 602 and reference marker 606 may be configured to provide for communication between reference markers. For example, the reference markers including reference marker 606 may be configured to connect via light transmission to configure a group of the reference markers as a network, where the reference markers are configured to send and receive signals such as optical signals.

While some reference markers are shown and/or described has being wired reference markers, in some implementations, some reference markers may be stand-alone, wireless reference markers. To the particular number of wireless reference markers may vary, and the number will depend on the particular implementation.

In some implementations, reference marker housing 602 may be any suitable plastic or composite such as an acrylonitrile butadiene styrene (ABS) resin or other thermoplastic resin, etc. The particular type of material may vary, and will depend on the particular implementation.

In various implementations, reference markers are attached to support portions using mounting mechanisms. In various implementations, each mounting mechanism of a wearable article may include a through hole or reference marker passage for passing through at least a portion of a reference marker. A reference marker passage may be a through hole in a support portion, where the active reference marker passes through the through hole for mounting. In some implementations, a mounting mechanism may include other mechanical elements. Further example implementations of mounting mechanisms are described below in connection with FIGS. 7A and 7B.

FIG. 7A is an exploded diagram of an example mounting mechanism assembly 700, according to some implementations. Shown is a fastener 702, a reference marker passage 704, and a reference marker housing 706. As shown, a dome portion 708 of reference marker housing 706 passes through reference marker passage 704. As indicated above, reference marker passage 704 may be a hole in a support portion of the wearable article. A base portion 710 may function as a stop. For example, in various implementations, base portion 710 has a larger diameter than reference marker passage 704. As such, as reference marker housing 706 passes through reference marker passage 704, base portion 710 is blocked from passing entirely through reference marker passage 704. In various implementations, reference marker housing 706 includes a wire passage 712 that functions to enable a wire (not shown) to pass through and connect to a reference marker (not shown). An example of a wire passing through a reference marker housing is shown in FIG. 6.

In various implementations, one or more of the mounting mechanism of the one or more mounting mechanisms may include a fastener 702 for securing a reference marker. In various implementations, fastener 702 may include or may be a securing ring or collar that is placed over reference marker housing 704. In various implementations, the securing ring secures the reference marker to the wearable article. In some implementations, the connection passages and associated cables also provide support to help keep the marker planer with the material of the wearable article. As described in more detail herein, the associated cables are conductive and provide electrical and/or optical conductivity of signals or any other form of conveying digital or analog information from one point to another (e.g., from a reference marker to a control unit, etc.). In practice, fastener 702 is placed over dome portion 708 and sandwiches or secures the material of the wearable article (not shown) between fastener 702 and base portion 710. In various implementations, the securing ring includes a predetermined color selected for imaging identification. For example, colors of some securing rings may be of a predetermined color to be identified and distinguished from other securing rings having other predetermined colors. The predetermined colors may vary and depend on the particular implementation.

FIG. 7B is a diagram of mounting mechanism assembly 700, where fastener 702 is coupled to the reference marker housing according to some implementations.

FIG. 8 is a rear perspective-view diagram of an example wearable article 800 for a performance capture system, which may be used for implementations described herein. Shown are a wire 802, markers 804, 808, 810, and a control box 812 or control unit 812. In various implementations, one or more of the support portions may include one or more connection conduits for connecting at least one reference marker to other reference markers or to control unit 812 via wires such as wire 802. In various implementations, connection conduits and wires connect between two or more of the reference marker passages in order to connect between two or more respective reference markers and/or control unit 812. In various implementations, each connection conduit receives an electrical wire such as wire 802 that is connected to at least one of the active markers.

As shown, the connections such as wire 802 may be curved and flexible in order to optimally follow the anatomy of the user. While being flexible, wire 802 may maintains some rigidity within a predefined range in order to maintain shape. In various implementations, the one or more connection conduits may be curved. Such curved connection conduits may guide and/or enable wires connecting reference markers to each other and/or to a control unit to be curved.

In various implementations, control unit 812 receives external signals (synchronizing signals, clock signals, etc.) from a mocap system (see FIG. 8). In some implementations, control unit 812 may make a copy of a clock signal, turn markers on and off, set the brightness, set the exposure, read the battery status, detect issues with reference markers, etc. Control unit 812 may also be referred to as a body pack.

Although the steps, operations, or computations may be presented in a specific order, the order may be changed in particular implementations. Other orderings of the steps are possible, depending on the particular implementation. In some particular implementations, multiple steps shown as sequential in this specification may be performed at the same time. Also, some implementations may not have all of the steps shown and/or may have other steps instead of, or in addition to, those shown herein.

Implementations described herein provide various benefits. For example, implementations provide a functional, reliable, and comfortable wearable article for a performance capture system.

The following implementations may be used to implement a performance capture system, including exchanging signals (e.g., sending and/or receiving signals) between the performance capture system and a wearable article. Such signals may include control signals and data signals, for example. In various implementations, control signals may be used by the performance capture system to control individual references markers. Also, in various implementations, the performance capture system and individual references markers may exchange data signals.

FIG. 9 is a block diagram of an exemplary computer system 900, which may be used for implementations described herein. Computer system 900 is merely illustrative and not intended to limit the scope of the claims. One of ordinary skill in the art would recognize other variations, modifications, and alternatives. For example, computer system 900 may be implemented in a distributed client-server configuration having one or more client devices in communication with one or more server systems.

In one exemplary implementation, computer system 900 includes a display device such as a monitor 910, computer 920, a data entry interface 930 such as a keyboard, touch device, and the like, a user input device 940, a network communication interface 950, and the like. User input device 940 is typically embodied as a computer mouse, a trackball, a track pad, wireless remote, tablet, touch screen, and the like. Moreover, user input device 940 typically allows a user to select and operate objects, icons, text, characters, and the like that appear, for example, on the monitor 910.

Network interface 950 typically includes an Ethernet card, a modem (telephone, satellite, cable, ISDN), (asynchronous) digital subscriber line (DSL) unit, and the like. Further, network interface 950 may be physically integrated on the motherboard of computer 920, may be a software program, such as soft DSL, or the like.

Computer system 900 may also include software that enables communications over communication network 952 such as the HTTP, TCP/IP, RTP/RTSP, protocols, wireless application protocol (WAP), IEEE 902.11 protocols, and the like. In addition to and/or alternatively, other communications software and transfer protocols may also be used, for example IPX, UDP or the like. Communication network 952 may include a local area network, a wide area network, a wireless network, an Intranet, the Internet, a private network, a public network, a switched network, or any other suitable communication network, such as for example Cloud networks. Communication network 952 may include many interconnected computer systems and any suitable communication links such as hardwire links, optical links, satellite or other wireless communications links such as BLUETOOTH, WIFI, wave propagation links, or any other suitable mechanisms for communication of information. For example, communication network 952 may communicate to one or more mobile wireless devices 956A-N, such as mobile phones, tablets, and the like, via a base station such as wireless transceiver 954.

Computer 920 typically includes familiar computer components such as a processor 960, and memory storage devices, such as a memory 970, e.g., random access memory (RAM), storage media 980, and system bus 990 interconnecting the above components. In one implementation, computer 920 is a PC compatible computer having multiple microprocessors, graphics processing units (GPU), and the like. While a computer is shown, it will be readily apparent to one of ordinary skill in the art that many other hardware and software configurations are suitable for use with the present invention. Memory 970 and Storage media 980 are examples of tangible non-transitory computer readable media for storage of data, audio/video files, computer programs, and the like. Other types of tangible media include disk drives, solid-state drives, floppy disks, optical storage media and bar codes, semiconductor memories such as flash drives, flash memories, random-access or read-only types of memories, battery-backed volatile memories, networked storage devices, cloud storage, and the like.

FIG. 10 is a block diagram of an example visual content generation system 1000, which may be used to generate imagery in the form of still images and/or video sequences of images, according to some implementations. The visual content generation system 1000 might generate imagery of live action scenes, computer generated scenes, or a combination thereof. In a practical system, users are provided with tools that allow them to specify, at high levels and low levels where necessary, what is to go into that imagery. For example, a user might be an animation artist and might use the visual content generation system 1000 to capture interaction between two human actors performing live on a sound stage and replace one of the human actors with a computer-generated anthropomorphic non-human being that behaves in ways that mimic the replaced human actor's movements and mannerisms, and then add in a third computer-generated character and background scene elements that are computer-generated, all in order to tell a desired story or generate desired imagery.

Still images that are output by the visual content generation system 1000 might be represented in computer memory as pixel arrays, such as a two-dimensional array of pixel color values, each associated with a pixel having a position in a two-dimensional image array. Pixel color values might be represented by three or more (or fewer) color values per pixel, such as a red value, a green value, and a blue value (e.g., in RGB format). Dimensions of such a two-dimensional array of pixel color values might correspond to a preferred and/or standard display scheme, such as 1920 pixel columns by 1280 pixel rows. Images might or might not be stored in a compressed format, but either way, a desired image may be represented as a two-dimensional array of pixel color values. In another variation, images are represented by a pair of stereo images for three-dimensional presentations and in other variations, some or all of an image output might represent three-dimensional imagery instead of just two-dimensional views.

A stored video sequence might include a plurality of images such as the still images described above, but where each image of the plurality of images has a place in a timing sequence, and the stored video sequence is arranged so that when each image is displayed in order, at a time indicated by the timing sequence, the display presents what appears to be moving and/or changing imagery. In one representation, each image of the plurality of images is a video frame having a specified frame number that corresponds to an amount of time that would elapse from when a video sequence begins playing until that specified frame is displayed. A frame rate might be used to describe how many frames of the stored video sequence are displayed per unit time. Example video sequences might include 24 frames per second (24 FPS), 50 FPS, 80 FPS, or other frame rates. In some implementations, frames are interlaced or otherwise presented for display, but for the purpose of clarity of description, in some examples, it is assumed that a video frame has one specified display time and it should be understood that other variations are possible.

One method of creating a video sequence is to simply use a video camera to record a live action scene, i.e., events that physically occur and can be recorded by a video camera. The events being recorded can be events to be interpreted as viewed (such as seeing two human actors talk to each other) and/or can include events to be interpreted differently due to clever camera operations (such as moving actors about a stage to make one appear larger than the other despite the actors actually being of similar build, or using miniature objects with other miniature objects so as to be interpreted as a scene containing life-sized objects).

Creating video sequences for story-telling or other purposes often calls for scenes that cannot be created with live actors, such as a talking tree, an anthropomorphic object, space battles, and the like. Such video sequences might be generated computationally rather than capturing light from live scenes. In some instances, an entirety of a video sequence might be generated computationally, as in the case of a computer-animated feature film. In some video sequences, it is desirable to have some computer-generated imagery and some live action, perhaps with some careful merging of the two.

While computer-generated imagery might be creatable by manually specifying each color value for each pixel in each frame, this is likely too tedious to be practical. As a result, a creator uses various tools to specify the imagery at a higher level. As an example, an artist might specify the positions in a scene space, such as a three-dimensional coordinate system, might specify positions of objects and/or lighting, as well as a camera viewpoint, and a camera view plane. Taking all of those as inputs, a rendering engine may compute each of the pixel values in each of the frames. In another example, an artist specifies position and movement of an articulated object having some specified texture rather than specifying the color of each pixel representing that articulated object in each frame.

In a specific example, a rendering engine performs ray tracing where a pixel color value is determined by computing which objects lie along a ray traced in the scene space from the camera viewpoint through a point or portion of the camera view plane that corresponds to that pixel. For example, a camera view plane might be represented as a rectangle having a position in the scene space that is divided into a grid corresponding to the pixels of the ultimate image to be generated. In the example, a ray defined by the camera viewpoint in the scene space and a given pixel in that grid first intersects a solid, opaque, blue object, that a given pixel is assigned the color blue. Of course, for modern computer-generated imagery, determining pixel colors, and thereby generating imagery, can be more complicated, as there are lighting issues, reflections, interpolations, and other considerations.

In various implementations, a live action capture system 1002 captures a live scene that plays out on a stage 1004. The live action capture system 1002 is described herein in greater detail, but might include computer processing capabilities, image processing capabilities, one or more processors, program code storage for storing program instructions executable by the one or more processors, as well as user input devices and user output devices, not all of which are shown.

In a specific live action capture system, cameras 1006(1) and 1006(2) capture the scene, while in some systems, there might be other sensor(s) 1008 that capture information from the live scene (e.g., infrared cameras, infrared sensors, motion capture (“mo-cap”) detectors, etc.). On the stage 1004, there might be human actors, animal actors, inanimate objects, background objects, and possibly an object such as a green screen 1010 that is designed to be captured in a live scene recording in such a way that it is easily overlaid with computer-generated imagery. The stage 1004 might also contain objects that serve as fiducials, such as fiducials 1012(1)-(3), that might be used post-capture to determine where an object was during capture. A live action scene might be illuminated by one or more lights, such as an overhead light 1014.

During or following the capture of a live action scene, the live action capture system 1002 might output live action footage to a live action footage storage 1020. A live action processing system 1022 might process live action footage to generate data about that live action footage and store that data into a live action metadata storage 1024. The live action processing system 1022 might include computer processing capabilities, image processing capabilities, one or more processors, program code storage for storing program instructions executable by the one or more processors, as well as user input devices and user output devices, not all of which are shown. The live action processing system 1022 might process live action footage to determine boundaries of objects in a frame or multiple frames, determine locations of objects in a live action scene, where a camera was relative to some action, distances between moving objects and fiducials, etc. Where elements are detected by sensor or other means, the metadata might include location, color, and intensity of the overhead light 1014, as that might be useful in post-processing to match computer-generated lighting on objects that are computer-generated and overlaid on the live action footage. The live action processing system 1022 might operate autonomously, perhaps based on predetermined program instructions, to generate and output the live action metadata upon receiving and inputting the live action footage. The live action footage can be camera-captured data as well as data from other sensors.

An animation creation system 1030 is another part of the visual content generation system 1000. The animation creation system 1030 might include computer processing capabilities, image processing capabilities, one or more processors, program code storage for storing program instructions executable by the one or more processors, as well as user input devices and user output devices, not all of which are shown. The animation creation system 1030 might be used by animation artists, managers, and others to specify details, perhaps programmatically and/or interactively, of imagery to be generated. From user input and data from a database or other data source, indicated as a data store 1032, the animation creation system 1030 might generate and output data representing objects (e.g., a horse, a human, a ball, a teapot, a cloud, a light source, a texture, etc.) to an object storage 1034, generate and output data representing a scene into a scene description storage 1036, and/or generate and output data representing animation sequences to an animation sequence storage 1038.

Scene data might indicate locations of objects and other visual elements, values of their parameters, lighting, camera location, camera view plane, and other details that a rendering engine 1050 might use to render CGI imagery. For example, scene data might include the locations of several articulated characters, background objects, lighting, etc. specified in a two-dimensional space, three-dimensional space, or other dimensional space (such as a 2.5-dimensional space, three-quarter dimensions, pseudo-3D spaces, etc.) along with locations of a camera viewpoint and view place from which to render imagery. For example, scene data might indicate that there is to be a red, fuzzy, talking dog in the right half of a video and a stationary tree in the left half of the video, all illuminated by a bright point light source that is above and behind the camera viewpoint. In some cases, the camera viewpoint is not explicit, but can be determined from a viewing frustum. In the case of imagery that is to be rendered to a rectangular view, the frustum would be a truncated pyramid. Other shapes for a rendered view are possible and the camera view plane could be different for different shapes.

The animation creation system 1030 might be interactive, allowing a user to read in animation sequences, scene descriptions, object details, etc. and edit those, possibly returning them to storage to update or replace existing data. As an example, an operator might read in objects from object storage into a baking processor that would transform those objects into simpler forms and return those to the object storage 1034 as new or different objects. For example, an operator might read in an object that has dozens of specified parameters (movable joints, color options, textures, etc.), select some values for those parameters and then save a baked object that is a simplified object with now fixed values for those parameters.

Rather than have to specify each detail of a scene, data from the data store 1032 might be used to drive object presentation. For example, if an artist is creating an animation of a spaceship passing over the surface of the Earth, instead of manually drawing or specifying a coastline, the artist might specify that the animation creation system 1030 is to read data from the data store 1032 in a file containing coordinates of Earth coastlines and generate background elements of a scene using that coastline data.

Animation sequence data might be in the form of time series of data for control points of an object that has attributes that are controllable. For example, an object might be a humanoid character with limbs and joints that are movable in manners similar to typical human movements. An artist can specify an animation sequence at a high level, such as “the left hand moves from location (X1, Y1, Z1) to (X2, Y2, Z2) over time T1 to T2”, at a lower level (e.g., “move the elbow joint 2.5 degrees per frame”) or even at a very high level (e.g., “character A should move, consistent with the laws of physics that are given for this scene, from point P1 to point P2 along a specified path”).

Animation sequences in an animated scene might be specified by what happens in a live action scene. An animation driver generator 1044 might read in live action metadata, such as data representing movements and positions of body parts of a live actor during a live action scene, and generate corresponding animation parameters to be stored in the animation sequence storage 1038 for use in animating a CGI object. This can be useful where a live action scene of a human actor is captured while wearing mo-cap fiducials (e.g., high-contrast markers outside actor clothing, high-visibility paint on actor skin, face, etc.) and the movement of those fiducials is determined by the live action processing system 1022. The animation driver generator 1044 might convert that movement data into specifications of how joints of an articulated CGI character are to move over time.

A rendering engine 1050 can read in animation sequences, scene descriptions, and object details, as well as rendering engine control inputs, such as a resolution selection and a set of rendering parameters. Resolution selection might be useful for an operator to control a trade-off between speed of rendering and clarity of detail, as speed might be more important than clarity for a movie maker to test a particular interaction or direction, while clarity might be more important than speed for a movie maker to generate data that will be used for final prints of feature films to be distributed. The rendering engine 1050 might include computer processing capabilities, image processing capabilities, one or more processors, program code storage for storing program instructions executable by the one or more processors, as well as user input devices and user output devices, not all of which are shown.

The visual content generation system 1000 can also include a merging system 1060 (labeled “Live +CGI Merging System”) that merges live footage with animated content. The live footage might be obtained and input by reading from the live action footage storage 1020 to obtain live action footage, by reading from the live action metadata storage 1024 to obtain details such as presumed segmentation in captured images segmenting objects in a live action scene from their background (perhaps aided by the fact that the green screen 1010 was part of the live action scene), and by obtaining CGI imagery from the rendering engine 1050.

A merging system 1060 might also read data from rule sets for merging/combining storage 1062. A very simple example of a rule in a rule set might be “obtain a full image including a two-dimensional pixel array from live footage, obtain a full image including a two-dimensional pixel array from the rendering engine 1050, and output an image where each pixel is a corresponding pixel from the rendering engine 1050 when the corresponding pixel in the live footage is a specific color of green, otherwise output a pixel value from the corresponding pixel in the live footage.”

The merging system 1060 might include computer processing capabilities, image processing capabilities, one or more processors, program code storage for storing program instructions executable by the one or more processors, as well as user input devices and user output devices, not all of which are shown. The merging system 1060 might operate autonomously, following programming instructions, or might have a user interface or programmatic interface over which an operator can control a merging process. In some implementations, an operator can specify parameter values to use in a merging process and/or might specify specific tweaks to be made to an output of the merging system 1060, such as modifying boundaries of segmented objects, inserting blurs to smooth out imperfections, or adding other effects. Based on its inputs, the merging system 1060 can output an image to be stored in a static image storage 1070 and/or a sequence of images in the form of video to be stored in an animated/combined video storage 1072.

Thus, as described, the visual content generation system 1000 can be used to generate video that combines live action with computer-generated animation using various components and tools, some of which are described in more detail herein. While the visual content generation system 1000 might be useful for such combinations, with suitable settings, it can be used for outputting entirely live action footage or entirely CGI sequences. The code may also be provided and/or carried by a transitory computer readable medium, e.g., a transmission medium such as in the form of a signal transmitted over a network.

According to one implementation, the techniques described herein are implemented by one or more generalized computing systems programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Special-purpose computing devices may be used, such as desktop computer systems, portable computer systems, handheld devices, networking devices or any other device that incorporates hard-wired and/or program logic to implement the techniques.

FIG. 11 is a block diagram of an example computer system 1100, which may be used for implementations described herein. The computer system 1100 includes a bus 1102 or other communication mechanism for communicating information, and a processor 1104 coupled with the bus 1102 for processing information. The processor 1104 may be, for example, a general-purpose microprocessor.

The computer system 1100 also includes a main memory 1106, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 1102 for storing information and instructions to be executed by the processor 1104. The main memory 1106 may also be used for storing temporary variables or other intermediate information during execution of instructions to be executed by the processor 1104. Such instructions, when stored in non-transitory storage media accessible to the processor 1104, render the computer system 1100 into a special-purpose machine that is customized to perform the operations specified in the instructions.

The computer system 1100 further includes a read only memory (ROM) 1108 or other static storage device coupled to the bus 1102 for storing static information and instructions for the processor 1104. A storage device 1110, such as a magnetic disk or optical disk, is provided and coupled to the bus 1102 for storing information and instructions.

The computer system 1100 may be coupled via the bus 1102 to a display 1112, such as a computer monitor, for displaying information to a computer user. An input device 1114, including alphanumeric and other keys, is coupled to the bus 1102 for communicating information and command selections to the processor 1104. Another type of user input device is a cursor control 1116, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to the processor 1104 and for controlling cursor movement on the display 1112. This input device 1114 typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the input device 1114 to specify positions in a plane.

The computer system 1100 may implement the techniques described herein using customized hard-wired logic, one or more ASICs or FPGAs, firmware, and/or program logic, which, in combination with the computer system, causes or programs the computer system 1100 to be a special-purpose machine. According to one implementation, the techniques herein are performed by the computer system 1100 in response to the processor 1104 executing one or more sequences of one or more instructions contained in the main memory 1106. Such instructions may be read into the main memory 1106 from another storage medium, such as the storage device 1110. Execution of the sequences of instructions contained in the main memory 1106 causes the processor 1104 to perform the process steps described herein. In alternative implementations, hard-wired circuitry may be used in place of or in combination with software instructions.

The term “storage media” as used herein refers to any non-transitory media that store data and/or instructions that cause a machine to operate in a specific fashion. Such storage media may include non-volatile media and/or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as the storage device 1110. Volatile media includes dynamic memory, such as the main memory 1106. Common forms of storage media include, for example, a floppy disk, a flexible disk, hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge.

Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media. For example, transmission media includes coaxial cables, copper wire, and fiber optics, including the wires that include the bus 1102. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

Various forms of media may be involved in carrying one or more sequences of one or more instructions to the processor 1104 for execution. For example, the instructions may initially be carried on a magnetic disk or solid-state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a network connection. A modem or network interface local to the computer system 1100 can receive the data. The bus 1102 carries the data to the main memory 1106, from which the processor 1104 retrieves and executes the instructions. The instructions received by the main memory 1106 may optionally be stored on the storage device 1110 either before or after execution by the processor 1104.

The computer system 1100 also includes a communication interface 1118 coupled to the bus 1102. The communication interface 1118 provides a two-way data communication coupling to a network link 1120 that is connected to a local network 1122. For example, the communication interface 1118 may be an integrated services digital network (“ISDN”) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. Wireless links may also be implemented. In any such implementation, the communication interface 1118 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

The network link 1120 typically provides data communication through one or more networks to other data devices. For example, the network link 1120 may provide a connection through a local network 1122 to a host computer 1124 or to data equipment operated by an Internet Service Provider (“ISP”) 1126. The ISP 1126 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the “Internet” 1128. The local network 1122 and the Internet 1128 both use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and the signals on the network link 1120 and through the communication interface 1118, which carry the digital data to and from the computer system 1100, are example forms of transmission media.

The computer system 1100 can send messages and receive data, including program code, through the network(s), the network link 1120, and the communication interface 1118. In the Internet example, a server 1130 might transmit a requested code for an application program through the Internet 1128, the ISP 1126, the local network 1122, and the communication interface 1118. The received code may be executed by the processor 1104 as it is received, and/or stored in the storage device 1110, or other non-volatile storage for later execution.

Operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. Processes described herein may be performed under the control of one or more computer systems (e.g., the computer system 1100) configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executing collectively on one or more processors, by hardware, or combinations thereof. The code may be stored on a computer-readable storage medium, for example, in the form of a computer program including a plurality of instructions executable by one or more processors. The computer-readable storage medium may be non-transitory. The code may also be carried by any computer-readable carrier medium, such as a transient medium or signal, e.g., a signal transmitted over a communications network.

Although the description has been described with respect to particular implementations thereof, these particular implementations are merely illustrative, and not restrictive. Controls can be provided to allow modifying various parameters of the compositing at the time of performing the recordings. For example, the resolution, number of frames, accuracy of depth position may all be subject to human operator changes or selection.

Any suitable programming language can be used to implement the routines of particular implementations including C, C++, Java, assembly language, etc. Different programming techniques can be employed such as procedural or object oriented. The routines can execute on a single processing device or multiple processors. Although the steps, operations, or computations may be presented in a specific order, this order may be changed in different particular implementations. In some particular implementations, multiple steps shown as sequential in this specification can be performed at the same time.

Particular implementations may be implemented in a computer-readable storage medium for use by or in connection with the instruction execution system, apparatus, system, or device. Particular implementations can be implemented in the form of control logic in software or hardware or a combination of both. The control logic, when executed by one or more processors, may be operable to perform that which is described in particular implementations.

Some implementations may be implemented as a system that includes one or more processors and a storage medium storing processor-readable instructions. When executed by the one or more processors of the system, the processor-readable instructions cause the system to carry out implementations described herein.

Some implementations may be implemented as a carrier medium carrying computer-readable code. When executed by one or more processors of a computer, the computer-readable code causes the computer to carry out implementations described herein.

Some implementations may be implemented as a non-transitory computer-readable storage medium storing computer-readable code. When executed by one or more processors of a computer, the computer-readable code causes the computer to carry out implementations described herein.

Some implementations may be implemented as a non-transitory processor-readable medium including instructions executable by one or more digital processors. The processor-readable medium includes one or more processor-readable instructions executable by the one or more digital processors for implementing implementations described herein.

Some implementations may be implemented as processor-implementable code provided on a computer-readable medium. The computer-readable medium may include a non-transient storage medium, such as solid-state memory, a magnetic disk, optical disk, etc., or a transient medium such as a signal transmitted over a computer network. When executed by one or more processors of a computer, the processor- implementable code causes the computer to carry out implementations described herein.

Particular implementations may be implemented by using a programmed general purpose digital computer, by using application specific integrated circuits, programmable logic devices, field programmable gate arrays, optical, chemical, biological, quantum or nanoengineered systems, components and mechanisms may be used. In general, the functions of particular implementations can be achieved by any means as is known in the art. Distributed, networked systems, components, and/or circuits can be used. Communication, or transfer, of data may be wired, wireless, or by any other means.

It will also be appreciated that one or more of the elements depicted in the drawings/figures can also be implemented in a more separated or integrated manner, or even removed or rendered as inoperable in certain cases, as is useful in accordance with a particular application. It is also within the spirit and scope to implement a program or code that can be stored in a machine-readable medium to permit a computer to perform any of the methods described above.

As used in the description herein and throughout the claims that follow, “a”, “an”, and “the” includes plural references unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the claims that follow, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise.

Thus, while particular implementations have been described herein, latitudes of modification, various changes, and substitutions are intended in the foregoing disclosures, and it will be appreciated that in some instances some features of particular implementations will be employed without a corresponding use of other features without departing from the scope and spirit as set forth. Therefore, many modifications may be made to adapt a particular situation or material to the essential scope and spirit. 

We claim:
 1. A wearable article for a performance capture system, the wearable article comprising: one or more regions, wherein the one or more regions are configured to be worn on at least a portion of a body of a user, and wherein at least one of the one or more regions are configured to hold performance capture equipment in predetermined positions; a plurality of mounting mechanisms coupled to the one or more regions for mounting reference markers to be used for position determination; and a plurality of fastening mechanisms coupled to the one or more regions for fastening devices and accessories for controlling the reference markers.
 2. The wearable article of claim 1, wherein the one or more regions comprise one or more torso portions, one or more arm portions, and one or more leg portions, and wherein the plurality of mounting mechanisms are coupled to one or more of the torso portions, arm portions, and leg portions.
 3. The wearable article of claim 1, wherein the one or more regions comprise one or more torso portions, and wherein the plurality of fastening mechanisms are coupled to one or more of the torso portions.
 4. The wearable article of claim 1, wherein at least one fastening mechanism comprises: a flap having a first edge and a second edge, wherein the first edge is fixed to a first region of the one or more regions, wherein the second edge is detachably coupled to a second region of the one or more regions, and wherein the second edge is detachable to enable the user to put on and take off the wearable article; and a strap for removably fastening at least one controller device for controlling the reference markers.
 5. The wearable article of claim 1, wherein the performance capture equipment comprises the reference markers, the devices, and the accessories.
 6. The wearable article of claim 1, wherein the one or more reference markers comprise one or more active reference markers.
 7. The wearable article of claim 1, wherein the one or more reference markers comprise one or more passive reference markers.
 8. A system for facilitating a performance capture system, the system comprising: one or more processors; and logic encoded in one or more non-transitory computer-readable storage media for execution by the one or more processors and when executed operable to cause the one or more processors to perform operations comprising: one or more regions, wherein the one or more regions are configured to be worn on at least a portion of a body of a user, and wherein at least one of the one or more regions are configured to hold performance capture equipment in predetermined positions; a plurality of mounting mechanisms coupled to the one or more regions for mounting reference markers to be used for position determination; and a plurality of fastening mechanisms coupled to the one or more regions for fastening devices and accessories for controlling the reference markers.
 9. The system of claim 8, wherein the one or more regions comprise one or more torso portions, one or more arm portions, and one or more leg portions, and wherein the plurality of mounting mechanisms are coupled to one or more of the torso portions, arm portions, and leg portions.
 10. The system of claim 8, wherein the one or more regions comprise one or more torso portions, and wherein the plurality of fastening mechanisms are coupled to one or more of the torso portions.
 11. The system of claim 8, wherein at least one fastening mechanism comprises: a flap having a first edge and a second edge, wherein the first edge is fixed to a first region of the one or more regions, wherein the second edge is detachably coupled to a second region of the one or more regions, and wherein the second edge is detachable to enable the user to put on and take off a wearable article associated with the system; and a strap for removably fastening at least one controller device for controlling the reference markers.
 12. The system of claim 8, wherein the performance capture equipment comprises the reference markers, the devices, and the accessories.
 13. The system of claim 8, wherein the one or more reference markers comprise one or more active reference markers.
 14. The system of claim 8, wherein the one or more reference markers comprise one or more passive reference markers.
 15. A non-transitory computer-readable storage medium with program instructions stored thereon, the program instructions when executed by one or more processors are operable to cause the one or more processors to perform operations comprising: one or more regions, wherein the one or more regions are configured to be worn on at least a portion of a body of a user, and wherein at least one of the one or more regions are configured to hold performance capture equipment in predetermined positions; a plurality of mounting mechanisms coupled to the one or more regions for mounting reference markers to be used for position determination; and a plurality of fastening mechanisms coupled to the one or more regions for fastening devices and accessories for controlling the reference markers.
 16. The computer-readable storage medium of claim 15, wherein the one or more regions comprise one or more torso portions, one or more arm portions, and one or more leg portions, and wherein the plurality of mounting mechanisms are coupled to one or more of the torso portions, arm portions, and leg portions.
 17. The computer-readable storage medium of claim 15, wherein the one or more regions comprise one or more torso portions, and wherein the plurality of fastening mechanisms are coupled to one or more of the torso portions.
 18. The computer-readable storage medium of claim 15, wherein at least one fastening mechanism comprises: a flap having a first edge and a second edge, wherein the first edge is fixed to a first region of the one or more regions, wherein the second edge is detachably coupled to a second region of the one or more regions, and wherein the second edge is detachable to enable the user to put on and take off a wearable article associated with the system; and a strap for removably fastening at least one controller device for controlling the reference markers.
 19. The computer-readable storage medium of claim 15, wherein the performance capture equipment comprises the reference markers, the devices, and the accessories.
 20. The computer-readable storage medium of claim 15, wherein the one or more reference markers comprise one or more active reference markers. 